{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Elasticsearch as a datastore\n",
    "\n",
    "In this walkthrough we will see how to use the retrieval API with a Elasticsearch datastore for *search / question-answering*.\n",
    "\n",
    "Before running this notebook you should have already initialized the retrieval API and have it running locally or elsewhere. See readme for instructions on how to do this."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## App Quickstart"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Install Python 3.10 if not already installed.\n",
    "\n",
    "2. Clone the `retrieval-app` repository:\n",
    "\n",
    "```\n",
    "git clone git@github.com:openai/retrieval-app.git\n",
    "```\n",
    "\n",
    "3. Navigate to the app directory:\n",
    "\n",
    "```\n",
    "cd /path/to/retrieval-app\n",
    "```\n",
    "\n",
    "4. Install `poetry`:\n",
    "\n",
    "```\n",
    "pip install poetry\n",
    "```\n",
    "\n",
    "5. Create a new virtual environment:\n",
    "\n",
    "```\n",
    "poetry env use python3.10\n",
    "```\n",
    "\n",
    "6. Install the `retrieval-app` dependencies:\n",
    "\n",
    "```\n",
    "poetry install\n",
    "```\n",
    "\n",
    "7. Set app environment variables:\n",
    "\n",
    "* `BEARER_TOKEN`: Secret token used by the app to authorize incoming requests. We will later include this in the request `headers`. The token can be generated however you prefer, such as using [jwt.io](https://jwt.io/).\n",
    "\n",
    "* `OPENAI_API_KEY`: The OpenAI API key used for generating embeddings with the OpenAI embeddings model. [Get an API key here](https://platform.openai.com/account/api-keys)!\n",
    "\n",
    "8. Set Elasticsearch-specific environment variables:\n",
    "\n",
    "* `DATASTORE`: set to `elasticsearch`.\n",
    "\n",
    "9. Set the Elasticsearch connection specific environment variables. Either set `ELASTICSEARCH_CLOUD_ID` or `ELASTICSEARCH_URL`.\n",
    "* `ELASTICSEARCH_CLOUD_ID`: Set to your deployment cloud id. You can find this in the [Elasticsearch console](https://cloud.elastic.co).\n",
    "\n",
    "* `ELASTICSEARCH_URL`: Set to your Elasticsearch URL, looks like `https://<username>:<password>@<host>:<port>`. You can find this in the [Elasticsearch console](https://cloud.elastic.co).\n",
    "\n",
    "10. Set the Elasticsearch authentication specific environment variables. Either set `ELASTICSEARCH_USERNAME` and `ELASTICSEARCH_PASSWORD` or `ELASTICSEARCH_API_KEY`.\n",
    "\n",
    "* `ELASTICSEARCH_USERNAME`: Set to your Elasticsearch username. You can find this in the [Elasticsearch console](https://cloud.elastic.co). Typically this is set to `elastic`.\n",
    "\n",
    "* `ELASTICSEARCH_PASSWORD`: Set to your Elasticsearch password. You can find this in the [Elasticsearch console](https://cloud.elastic.co) in security.\n",
    "\n",
    "* `ELASTICSEARCH_API_KEY`: Set to your Elasticsearch API key. You can set one up in Kibana Stack management page.\n",
    "\n",
    "11. Set the Elasticsearch index specific environment variables.\n",
    "\n",
    "* `ELASTICSEARCH_INDEX`: Set to the name of the Elasticsearch index you want to use.\n",
    "\n",
    "12. Run the app with:\n",
    "\n",
    "```\n",
    "poetry run start\n",
    "```\n",
    "\n",
    "If running the app locally you should see something like:\n",
    "\n",
    "```\n",
    "INFO:     Uvicorn running on http://0.0.0.0:8000\n",
    "INFO:     Application startup complete.\n",
    "```\n",
    "\n",
    "In that case, the app is automatically connected to our index (specified by `ELASTICSEARCH_INDEX`), if no index with that name existed beforehand, the app creates one for us.\n",
    "\n",
    "Now we're ready to move on to populating our index with some data."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Required Libraries"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a few Python libraries we must `pip install` for this notebook to run, those are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.2.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install -qU datasets pandas tqdm"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing Data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we will use the **S**tanford **Qu**estion **A**nswering **D**ataset (SQuAD2), which we download from Hugging Face Datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "\n",
    "data = load_dataset(\"squad_v2\", split=\"train\")\n",
    "data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transform the data into a Pandas dataframe for simpler preprocessing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>context</th>\n",
       "      <th>question</th>\n",
       "      <th>answers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>56be85543aeaaa14008c9063</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...</td>\n",
       "      <td>When did Beyonce start becoming popular?</td>\n",
       "      <td>{'text': ['in the late 1990s'], 'answer_start'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>56be85543aeaaa14008c9065</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...</td>\n",
       "      <td>What areas did Beyonce compete in when she was...</td>\n",
       "      <td>{'text': ['singing and dancing'], 'answer_star...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>56be85543aeaaa14008c9066</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...</td>\n",
       "      <td>When did Beyonce leave Destiny's Child and bec...</td>\n",
       "      <td>{'text': ['2003'], 'answer_start': [526]}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>56bf6b0f3aeaaa14008c9601</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...</td>\n",
       "      <td>In what city and state did Beyonce  grow up?</td>\n",
       "      <td>{'text': ['Houston, Texas'], 'answer_start': [...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>56bf6b0f3aeaaa14008c9602</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...</td>\n",
       "      <td>In which decade did Beyonce become famous?</td>\n",
       "      <td>{'text': ['late 1990s'], 'answer_start': [276]}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         id    title  \\\n",
       "0  56be85543aeaaa14008c9063  Beyoncé   \n",
       "1  56be85543aeaaa14008c9065  Beyoncé   \n",
       "2  56be85543aeaaa14008c9066  Beyoncé   \n",
       "3  56bf6b0f3aeaaa14008c9601  Beyoncé   \n",
       "4  56bf6b0f3aeaaa14008c9602  Beyoncé   \n",
       "\n",
       "                                             context  \\\n",
       "0  Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...   \n",
       "1  Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...   \n",
       "2  Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...   \n",
       "3  Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...   \n",
       "4  Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...   \n",
       "\n",
       "                                            question  \\\n",
       "0           When did Beyonce start becoming popular?   \n",
       "1  What areas did Beyonce compete in when she was...   \n",
       "2  When did Beyonce leave Destiny's Child and bec...   \n",
       "3      In what city and state did Beyonce  grow up?    \n",
       "4         In which decade did Beyonce become famous?   \n",
       "\n",
       "                                             answers  \n",
       "0  {'text': ['in the late 1990s'], 'answer_start'...  \n",
       "1  {'text': ['singing and dancing'], 'answer_star...  \n",
       "2          {'text': ['2003'], 'answer_start': [526]}  \n",
       "3  {'text': ['Houston, Texas'], 'answer_start': [...  \n",
       "4    {'text': ['late 1990s'], 'answer_start': [276]}  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = data.to_pandas()\n",
    "data.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset contains a lot of duplicate `context` paragraphs, this is because each `context` can have many relevant questions. We don't want these duplicates so we remove like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19029\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>context</th>\n",
       "      <th>question</th>\n",
       "      <th>answers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>56be85543aeaaa14008c9063</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...</td>\n",
       "      <td>When did Beyonce start becoming popular?</td>\n",
       "      <td>{'text': ['in the late 1990s'], 'answer_start'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>56be86cf3aeaaa14008c9076</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Following the disbandment of Destiny's Child i...</td>\n",
       "      <td>After her second solo album, what other entert...</td>\n",
       "      <td>{'text': ['acting'], 'answer_start': [207]}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>56be88473aeaaa14008c9080</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>A self-described \"modern-day feminist\", Beyonc...</td>\n",
       "      <td>In her music, what are some recurring elements...</td>\n",
       "      <td>{'text': ['love, relationships, and monogamy']...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>56be892d3aeaaa14008c908b</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé Giselle Knowles was born in Houston, T...</td>\n",
       "      <td>Beyonce's younger sibling also sang with her i...</td>\n",
       "      <td>{'text': ['Destiny's Child'], 'answer_start': ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>56be8a583aeaaa14008c9094</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>Beyoncé attended St. Mary's Elementary School ...</td>\n",
       "      <td>What town did Beyonce go to school in?</td>\n",
       "      <td>{'text': ['Fredericksburg'], 'answer_start': [...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          id    title  \\\n",
       "0   56be85543aeaaa14008c9063  Beyoncé   \n",
       "15  56be86cf3aeaaa14008c9076  Beyoncé   \n",
       "27  56be88473aeaaa14008c9080  Beyoncé   \n",
       "39  56be892d3aeaaa14008c908b  Beyoncé   \n",
       "52  56be8a583aeaaa14008c9094  Beyoncé   \n",
       "\n",
       "                                              context  \\\n",
       "0   Beyoncé Giselle Knowles-Carter (/biːˈjɒnseɪ/ b...   \n",
       "15  Following the disbandment of Destiny's Child i...   \n",
       "27  A self-described \"modern-day feminist\", Beyonc...   \n",
       "39  Beyoncé Giselle Knowles was born in Houston, T...   \n",
       "52  Beyoncé attended St. Mary's Elementary School ...   \n",
       "\n",
       "                                             question  \\\n",
       "0            When did Beyonce start becoming popular?   \n",
       "15  After her second solo album, what other entert...   \n",
       "27  In her music, what are some recurring elements...   \n",
       "39  Beyonce's younger sibling also sang with her i...   \n",
       "52             What town did Beyonce go to school in?   \n",
       "\n",
       "                                              answers  \n",
       "0   {'text': ['in the late 1990s'], 'answer_start'...  \n",
       "15        {'text': ['acting'], 'answer_start': [207]}  \n",
       "27  {'text': ['love, relationships, and monogamy']...  \n",
       "39  {'text': ['Destiny's Child'], 'answer_start': ...  \n",
       "52  {'text': ['Fredericksburg'], 'answer_start': [...  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = data.drop_duplicates(subset=[\"context\"])\n",
    "print(len(data))\n",
    "data.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The format required by the apps `upsert` function is a list of documents like:\n",
    "\n",
    "```json\n",
    "[\n",
    "    {\n",
    "        \"id\": \"abc\",\n",
    "        \"text\": \"some important document text\",\n",
    "        \"metadata\": {\n",
    "            \"field1\": \"optional metadata goes here\",\n",
    "            \"field2\": 54\n",
    "        }\n",
    "    },\n",
    "    {\n",
    "        \"id\": \"123\",\n",
    "        \"text\": \"some other important text\",\n",
    "        \"metadata\": {\n",
    "            \"field1\": \"another metadata\",\n",
    "            \"field2\": 71,\n",
    "            \"field3\": \"not all metadatas need the same structure\"\n",
    "        }\n",
    "    }\n",
    "    ...\n",
    "]\n",
    "```\n",
    "\n",
    "Every document *must* have a `\"text\"` field. The `\"id\"` and `\"metadata\"` fields are optional.\n",
    "\n",
    "To create this format for our SQuAD data we do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "documents = [\n",
    "    {\n",
    "        'id': r['id'],\n",
    "        'text': r['context'],\n",
    "        'metadata': {\n",
    "            'title': r['title']\n",
    "        }\n",
    "    } for r in data.to_dict(orient='records')\n",
    "]\n",
    "documents[:3]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Indexing the Docs"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, it's time to initiate the indexing process, also known as upserting, for our documents. To perform these requests to the retrieval app API, we must provide authorization using the BEARER_TOKEN we defined earlier. Below is how we accomplish this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "BEARER_TOKEN = os.environ.get(\"BEARER_TOKEN\") or \"BEARER_TOKEN_HERE\"\n",
    "\n",
    "headers = {\n",
    "    \"Authorization\": f\"Bearer {BEARER_TOKEN}\"\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will execute bulk inserts in batches set by the `batch_size`.\n",
    "\n",
    "Now that all our SQuAD2 records have been successfully indexed, we can proceed with the querying phase."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1/1 [00:16<00:00, 16.88s/it]\n"
     ]
    }
   ],
   "source": [
    "from tqdm.auto import tqdm\n",
    "import requests\n",
    "from requests.adapters import HTTPAdapter, Retry\n",
    "\n",
    "batch_size = 100\n",
    "endpoint_url = \"http://localhost:8000\"\n",
    "s = requests.Session()\n",
    "\n",
    "# we setup a retry strategy to retry on 5xx errors\n",
    "retries = Retry(\n",
    "    total=5,  # number of retries before raising error\n",
    "    backoff_factor=0.1,\n",
    "    status_forcelist=[500, 502, 503, 504]\n",
    ")\n",
    "s.mount('http://', HTTPAdapter(max_retries=retries))\n",
    "\n",
    "for i in tqdm(range(0, 10, batch_size)):\n",
    "    i_end = min(len(documents), i+batch_size)\n",
    "    # make post request that allows up to 5 retries\n",
    "    res = s.post(\n",
    "        f\"{endpoint_url}/upsert\",\n",
    "        headers=headers,\n",
    "        json={\n",
    "            \"documents\": documents[i:i_end]\n",
    "        }\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Making Queries"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By passing one or more queries to the /query endpoint, we can easily conduct a query on the datastore. For this task, we can utilize a few questions from SQuAD2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "19029"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "queries = data['question'].tolist()\n",
    "# format into the structure needed by the /query endpoint\n",
    "queries = [{'query': queries[i]} for i in range(len(queries))]\n",
    "len(queries)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [200]>"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res = requests.post(\n",
    "    \"http://0.0.0.0:8000/query\",\n",
    "    headers=headers,\n",
    "    json={\n",
    "        'queries': queries[:3]\n",
    "    }\n",
    ")\n",
    "res"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we have the ability to iterate through the responses and observe the outcomes obtained for each query:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------------\n",
      "When did Beyonce start becoming popular?\n",
      "\n",
      "0.93: On December 13, 2013, Beyoncé unexpectedly released her eponymous fifth studio album on the iTunes Store without any prior announcement or promotion. The album debuted atop the Billboard 200 chart, giving Beyoncé her fifth consecutive number-one album in the US. This made her the first woman in the chart's history to have her first five studio albums debut at number one. Beyoncé received critical acclaim and commercial success, selling one million digital copies worldwide in six days; The New York Times noted the album's unconventional, unexpected release as significant. Musically an electro-R&B album, it concerns darker themes previously unexplored in her work, such as \"bulimia, postnatal depression [and] the fears and insecurities of marriage and motherhood\". The single \"Drunk in Love\", featuring Jay Z, peaked at number two on the Billboard Hot 100 chart.\n",
      "0.93: Beyoncé's first solo recording was a feature on Jay Z's \"'03 Bonnie & Clyde\" that was released in October 2002, peaking at number four on the U.S. Billboard Hot 100 chart. Her first solo album Dangerously in Love was released on June 24, 2003, after Michelle Williams and Kelly Rowland had released their solo efforts. The album sold 317,000 copies in its first week, debuted atop the Billboard 200, and has since sold 11 million copies worldwide. The album's lead single, \"Crazy in Love\", featuring Jay Z, became Beyoncé's first number-one single as a solo artist in the US. The single \"Baby Boy\" also reached number one, and singles, \"Me, Myself and I\" and \"Naughty Girl\", both reached the top-five.\n",
      "0.93: Beyoncé is believed to have first started a relationship with Jay Z after a collaboration on \"'03 Bonnie & Clyde\", which appeared on his seventh album The Blueprint 2: The Gift & The Curse (2002). Beyoncé appeared as Jay Z's girlfriend in the music video for the song, which would further fuel speculation of their relationship. On April 4, 2008, Beyoncé and Jay Z were married without publicity. As of April 2014, the couple have sold a combined 300 million records together. The couple are known for their private relationship, although they have appeared to become more relaxed in recent years. Beyoncé suffered a miscarriage in 2010 or 2011, describing it as \"the saddest thing\" she had ever endured. She returned to the studio and wrote music in order to cope with the loss.\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "After her second solo album, what other entertainment venture did Beyonce explore?\n",
      "\n",
      "0.93: Following the disbandment of Destiny's Child in June 2005, she released her second solo album, B'Day (2006), which contained hits \"Déjà Vu\", \"Irreplaceable\", and \"Beautiful Liar\". Beyoncé also ventured into acting, with a Golden Globe-nominated performance in Dreamgirls (2006), and starring roles in The Pink Panther (2006) and Obsessed (2009). Her marriage to rapper Jay Z and portrayal of Etta James in Cadillac Records (2008) influenced her third album, I Am... Sasha Fierce (2008), which saw the birth of her alter-ego Sasha Fierce and earned a record-setting six Grammy Awards in 2010, including Song of the Year for \"Single Ladies (Put a Ring on It)\".\n",
      "0.92: Beyoncé announced a hiatus from her music career in January 2010, heeding her mother's advice, \"to live life, to be inspired by things again\". During the break she and her father parted ways as business partners. Beyoncé's musical break lasted nine months and saw her visit multiple European cities, the Great Wall of China, the Egyptian pyramids, Australia, English music festivals and various museums and ballet performances.\n",
      "0.92: Beyoncé took a hiatus from music in 2010 and took over management of her career; her fourth album 4 (2011) was subsequently mellower in tone, exploring 1970s funk, 1980s pop, and 1990s soul. Her critically acclaimed fifth studio album, Beyoncé (2013), was distinguished from previous releases by its experimental production and exploration of darker themes.\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "In her music, what are some recurring elements in them?\n",
      "\n",
      "0.91: Beyoncé's music is generally R&B, but she also incorporates pop, soul and funk into her songs. 4 demonstrated Beyoncé's exploration of 90s-style R&B, as well as further use of soul and hip hop than compared to previous releases. While she almost exclusively releases English songs, Beyoncé recorded several Spanish songs for Irreemplazable (re-recordings of songs from B'Day for a Spanish-language audience), and the re-release of B'Day. To record these, Beyoncé was coached phonetically by American record producer Rudy Perez.\n",
      "0.9: The feminism and female empowerment themes on Beyoncé's second solo album B'Day were inspired by her role in Dreamgirls and by singer Josephine Baker. Beyoncé paid homage to Baker by performing \"Déjà Vu\" at the 2006 Fashion Rocks concert wearing Baker's trademark mini-hula skirt embellished with fake bananas. Beyoncé's third solo album I Am... Sasha Fierce was inspired by Jay Z and especially by Etta James, whose \"boldness\" inspired Beyoncé to explore other musical genres and styles. Her fourth solo album, 4, was inspired by Fela Kuti, 1990s R&B, Earth, Wind & Fire, DeBarge, Lionel Richie, Teena Marie with additional influences by The Jackson 5, New Edition, Adele, Florence and the Machine, and Prince.\n",
      "0.9: She has received co-writing credits for most of the songs recorded with Destiny's Child and her solo efforts. Her early songs were personally driven and female-empowerment themed compositions like \"Independent Women\" and \"Survivor\", but after the start of her relationship with Jay Z she transitioned to more man-tending anthems such as \"Cater 2 U\". Beyoncé has also received co-producing credits for most of the records in which she has been involved, especially during her solo efforts. However, she does not formulate beats herself, but typically comes up with melodies and ideas during production, sharing them with producers.\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for query_result in res.json()['results']:\n",
    "    query = query_result['query']\n",
    "    answers = []\n",
    "    scores = []\n",
    "    for result in query_result['results']:\n",
    "        answers.append(result['text'])\n",
    "        scores.append(round(result['score'], 2))\n",
    "    print(\"-\"*70+\"\\n\"+query+\"\\n\\n\"+\"\\n\".join([f\"{s}: {a}\" for a, s in zip(answers, scores)])+\"\\n\"+\"-\"*70+\"\\n\\n\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The top results are all relevant as we would have hoped. We can see that the `score` is a measure of how relevant the document is to the query. The higher the score the more relevant the document is to the query."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "chatgpt-retrieval-plugin-S7h-2AWq-py3.10",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.3"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "1979a773a5778de9a5fa593a629dff0ab3c80c2563810d3e6a8dfb123dc01c7d"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
